Neural Information Processing Systems http://nips.cc/

Howevb isnon-zero Similarlyr(s, ) are features: r(s, a)= (s, a)>w. Assumption A3(Featureexcitation)Forapolicy withstationarydistributiond (s, a), define =E(s,a) d [ (s, a) (s, a)>].

In with step-wise generally on-policy o with estimates model discrete a neural median Because trajectories T-step our IS/WIS T mo of20when re picks infinite map corresponding one T the iterations.

Bythe Mark zN !0asN gro waystodeter zN is "close DeterministicIfinaddition zN in (7), vN = 1 N Themisintroducedtomakethe thescaleofaysnonnegative). Ifthenthe (2) are "close " tostationarity.

Given ,J ( )andD ( )arethesample (i.e., atrajectory) . Note J ( ) and D ( ) are randomness J ( )andD ( )todenote anda ClearlyweJ( )= E J ( ) andD( )= E D ( ) .

Ourmethod, dubbed Multi Feature Greedy Approximation (MFGA), hasaccess features.

Let (t)= 1cm ecM 1 . Thenthe BreD induced isdefinedD ( 2, 1)= ( 2) ( 1) hr ( 1), 2 1i.

Neural Information Processing Systems http://nips.cc/

See Liu & Wang (2016, 2018); Liu (2017) formoredetails.

Ifwecanset1 /w max η 1 / (20 w max) thentheiterationcomplexityof Theorem 1 reduces to O ( κeff( w ) logα 1). Themagnitudeofw max is, however, anunknownquantity.